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IN THE CLAIMS: 

The text of all pending claims are set forth below. Cancelled and withdrawn claims 
are indicated with claim number and status only. The claims as listed below show added text 
with underlining and deleted text with strikethrough . The status of each claim is indicated 
with one of (original), (currently amended), (previously amended), (cancelled), (withdrawn), 
(new), (previously added), (reinstated - formerly claim #), (previously reinstated), (re- 
presented - formerly dependent claim #) or, (previously re-presented). 

Please AMEND the claims in accordance with the following: 



1 . (CURRENTLY AMENDED) A robot system having an image processing 
function for determining orientation, or orientation and position of a robot operation on an 
object of detection among a plurality of objects , the system comprising: 

a robot; 

a first image capturing device; 

a memory storing reference models, each croatod basod on comprising an image of 
a reference object captured by said image capturing device in a different direction, and for 
each reference model storing information of the capturing direction of its associated image 
and information of an orientation of the robot with respect to the reference object, said 
reference object being the ob je ct of dotoction one of the plurality of objects or having a shape 
substantially identical to that of tho ob j ect of d e t e ct i on one of the plurality of objects ; and 

a processor to perform matching processing on the reference model images and an-a 
working image of the ob je ct of d e t e ction plurality of objects (captured by said first image 
capturing device) us i ng sa i d roforonco mode l s to select an image of one of the reference 
models whoso i maao of tho roforonco ob j ect matched with that matches the imago of tho 
object of detection in the working image , and to determine the orientation, or the orientation 
and position of the robot operation to be performed on the object of detection, the 
determining based on the selected image of the reference object, based on said one 
reference model and the information of its associated capturing direction, and based on the 
information of the orientation of the robot with respect to the reference object that is 
associated with said one reference model. 
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2. (PREVIOUSLY AMENDED) A robot system having an image processing 
function according to claim 1 , wherein said reference models are obtained from a part of the 
image data of the reference object. 

3. (ORIGINAL) A robot system having an image processing function according to 
claim 1 , wherein said reference models are obtained by processing the image data of the 
reference object. 

4. (ORIGINAL) A robot system having an image processing function according to 
claim 1, wherein said first image capturing device comprises a camera for capturing two- 
dimensional image data. 

5. (ORIGINAL) A robot system having an image processing function according to 
claim 4, wherein said image data of the reference object are captured by said camera from a 
predetermined distance. 

6. (PREVIOUSLY AMENDED) A robot system having an image processing 
function according to claim 1 , further comprising: 

a second image capturing device; wherein 

said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with respect 
to the object, and wherein 

said processor processes second image data captured by said second image 
capturing device to detect position and/or posture of the object with respect to said second 
image data capturing device. 

7. (PREVIOUSLY AMENDED) A robot system having an image processing 
function according to claim 1 further comprising: 

a second image capturing device for obtaining three-dimensional position; wherein 
said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with respect 
to the object, so that said second image data capturing device is directed to a characterizing 
portion of the object; and wherein 

said processor detects three-dimensional position and/or posture of the object based 
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on three-dimensional position of said characterizing portion obtained by said second image 
capturing device. 

8. (CURRENTLY AMENDED) A robot system having an image processing 
function according to cla i m 1 claim 6 , wherein said first image data capturing device is used 
as said second image data capturing device. 

9. (CURRENTLY AMENDED) A robot system having an image processing 
function according to c l a i m 1 claim 6 , wherein said second image capturing device comprises 
a three-dimensional visual sensor of spot-light scanning type capable of measuring distance 
between the sensor and an object. 

1 0. (CURRENTLY AMENDED) A robot system having an image processing 
function according to cla i m 1 claim 6 . wherein said second image data capturing device 
comprises a structured-light unit for irradiating a structured light on an object and capturing 
an image of the object including the irradiated light on the object. 

1 1 . (PREVIOUSLY AMENDED) A robot system having an image processing 
function according to claim 7, wherein said robot operation is an operation of picking up at 
least one object from a plurality of objects overlapped with each other. 

1 2. (CURRENTLY AMENDED) A robot system having an image processing 
function for determining orientation, or orientation and position of a robot operation on an 
object of detection among a plurality of objects of plural kinds , the system comprising: 

a robot; 

a first image capturing device; 

a memory storing reference models, each croatod based on comprising images of 
each of different kinds of reference objects captured by said first image capturing device, and 
storing indicia of the kinds respectively associated with said reference models, and 
information of a different orientation of the robot with respect to each of the different images 
of the reference object of each kind, each of said k i nd of reference objects potent i a ll y being 
one of the kinds of the plurality of objects th e ob je ct of d e t e ct i on or an ob je ct or having a 
shape substantially identical to thereto that of the ob je ct of d e tect i on ; and 
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a processor to perform matching processing on aft -the reference model image s and a 
working image of the ob i oct of det e ct i on plurality of objects of plural kinds (captured by said 
first image capturing device) us i ng said referenc e mod el s to select an image of one of said 
kinds of the reference models that matches the object of detection in the working image , and 
to determine the orientation, or the orientation and position of the robot operation to be 
performed on the object of detection, the determining based on the selected image of the 
roforonco object , based on said one reference model, based on the indicia of the kind 
associated with said one reference model and the information of the orientation of the robot 
with respect to the reference object associated with said one reference model of said one 
kind. 

1 3. (PREVIOUSLY AMENDED) A robot system having an image processing 
function according to claim 12, wherein said reference models are obtained from a part of the 
image data of the reference object. 

14. (ORIGINAL) A robot system having an image processing function according to 
claim 12, wherein said reference models are obtained by processing the image data of the 
reference object. 

15. (ORIGINAL) A robot system having an image processing function according to 
claim 12, wherein said first image capturing device comprises a camera for capturing two- 
dimensional image data. 

16. (ORIGINAL) A robot system having an image processing function according to 
claim 15, wherein said image data of the reference object are captured by said camera from 
a predetermined distance. 

1 7. (PREVIOUSLY AMENDED) A robot system having an image processing 
function according to claim 12, further comprising a second image capturing device, wherein 

said robot situates said second image data capturing device to have said determined 
orientation or to have said determined orientation and said determined position with respect 
to the object, and wherein 

said processor processes second image data captured by said second image 
capturing device to detect position and/or posture of the object with respect to said second 
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image data capturing device. 

18. (PREVIOUSLY AMENDED) A robot system having an image processing 
function according to claim 12, further comprising: 

a second image capturing device for obtaining three-dimensional position; wherein 
said robot situates said second image data capturing device to have said determined 

orientation or to have said determined orientation and said determined position with respect 

to the object, so that said second image data capturing device is directed to a characterizing 

portion of the object; and wherein 

said processor detects three-dimensional position and/or posture of the object based 

on three-dimensional position of said characterizing portion obtained by said second image 

capturing device. 

19. (CURRENTLY AMENDED) A robot system having an image processing 
function according to c l a i m 12 claim 17 . wherein said first image data capturing device is 
used as said second image data capturing device. 

20. (CURRENTLY AMENDED) A robot system having an image processing 
function according to cla i m 12 claim 17 , wherein said second image capturing device 
comprises a three-dimensional visual sensor of spot-light scanning type capable of 
measuring distance between the sensor and an object. 

21 . (CURRENTLY AMENDED) A robot system having an image processing 
function according to c l a i m 12 claim 17 . wherein said second image data capturing device 
comprises a structured-light unit for irradiating a structured light on an object and capturing 
an image of the object including the irradiated light on the object. 

22. (PREVIOUSLY AMENDED) A robot system having an image processing 
function according to claim 18, wherein said robot operation is an operation of picking up at 
least one object from a plurality of objects overlapped with each other. 

23. (WITHDRAWN, CURRENTLY AMENDED) A method for automatically 
determining an arrangement of a workpiece relative to a robot, tho robot compr i sing an 
i mag i ng dovico affixed to an arm, the method comprising: 
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storing reference images of the workpiece or an object so shaped (workpiece/object) 
and reference arrangement information indicating arrangements of the robot and 
workpiece/object relative to each other when the images were captured; 

from a known arrangement of the robot, capturing a working image of the workpiece 
among a plurality of randomly arranged workpieces with tfre-an imaging device; 

finding one of the reference images that has a closest match to the workpiece in the 
working image; and 

determining a known arrangement of the robot relative to the workpiece based on 
information indicating the known arrangement of the robot, and based on the reference 
arrangement information corresponding to the found reference image. 

24. (WITHDRAWN) A method according to claim 23, wherein reference images 
and reference arrangement information is obtained for workpieces/objects of different 
shapes, and wherein the finding comprises first determining that a reference image of one of 
the different shapes matches the working image of the workpiece, and then finding one 
reference image of the shape that best matches the working image. 

25. (WITHDRAWN) A method according to claim 23, wherein the robot is used to 
capture the reference images, and wherein the reference arrangement information 
represents arrangements of the robot when capturing the reference images. 

26. (WITHDRAWN) A method according to claim 23, wherein a second imaging 
device is affixed to the arm and is used to determine additional arrangement information 
used to determine the known arrangement of the robot relative to the workpiece. 

27. (NEW) A robot system having an image processing function according to 
claim 7, wherein said first image data capturing device is used as said second image data 
capturing device. 

28. (NEW) A robot system having an image processing function according to 
claim 7, wherein said second image capturing device comprises a three-dimensional visual 
sensor of spot-light scanning type capable of measuring distance between the sensor and an 
object. 
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29. (NEW) A robot system having an image processing function according to 
claim 7, wherein said second image data capturing device comprises a structured-light unit 
for irradiating a structured light on an object and capturing an image of the object including 
the irradiated light on the object. 

30. (NEW) A robot system having an image processing function according to 
claim 18, wherein said first image data capturing device is used as said second image data 
capturing device. 

31 . (NEW) A robot system having an image processing function according to 
claim 18, wherein said second image capturing device comprises a three-dimensional visual 
sensor of spot-light scanning type capable of measuring distance between the sensor and an 
object. 



32. (NEW) A robot system having an image processing function according to 
claim 18, wherein said second image data capturing device comprises a structured-light unit 
for irradiating a structured light on an object and capturing an image of the object including 
the irradiated light on the object. 
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